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eStream Server Component Framework 
Low Level Design 



Michael Beckmann 
September 4, 2000 



Functionality 

The Server Component Framework provides a common basis on which server compo- 
nents are implemented. The framework provides a number of services such as common 
server initiaUzation and configuration, messaging, state management, logging, and error 
handling. The component framework ties together many of the core utilities provided for 
the server components. 

The advantage of the framework is that heterogeneous server components can be man- 
aged in a consistent manner with the expectation that all server components will commu- 
nicate and behave consistently within the system. 

All server components with the exception of the web server will be built on top the 
Server Component Framework. To make use of the Server Component Framework, a 
specialized server component will need to extend the framework by implementing the 
methods high-lighted in gray. Implementing these interfaces makes the specialized server 
component "plug-able" within the framework. 



Command Line Utilities/Logging Utilities 



StopProcessing 



StartProcessing 



UpdateConfig 



StartServer 



State ' ^""fiS Tfiread IMessage 
Manager ' ^^'^^^^^ Utilities Service 




HandleError 



StopServer 



\ 



Database Connectivity Layer 
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The following table give a brief description of each of the routines that need to be spe- 
cialized by each server component to make it plug-able into the Sei-ver Framework: 



StartProcessing 


Specialized server component routine to request the server compo- 
nent to start processing work. 


StopProcessing 


Specialized routine to request the server component stop processing 
work and transition into an idle state 


UpdateConfig 


Specialized routine to dynamically update configurations while a 
component is either in the processing or idle state. 


HandleError 


Specialized routine to handle the occurrence of an error 



Server State Manager: 



At the heart of the server component framework is the Server State Manager. The server 
state manager is a set of interfaces that initiate and manage state changes within a server 
component. All Server components, by virtue of being built on top of the component 
framework, can be managed uniformly across a deployment. 

The Server State Manager implements a simple state machine that is shared between 
components. It manages the state transitions within the server component. Additionally, 
the state manager maintains current state information for each server component and logs 
state transition history in the event that a server component terminates unexpectedly. 

As specified above, each server component is required to implement a number of 
transition methods, with pre-defined signatures, which the state manager will execute 
when making a state transition. 

The following diagram shows the state diagram and the associated transitions: 
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~^ StoppedJ^— 



Processing^ 



/ StopProcessing \ /_ 
^ 



Idle ^ 



1\ j ^ / StartProcessing | |~7\' 
/ RaiseError 



Error J 



Message Service: 

The Server Component Framework depends on a message service which is used by the 
Server State Manager and Configuration Manager to communicate with the System 
Monitor. 

Vcvt Server State Manager uses the messaging service to hsten for state change requests 
from the System Monitor which it satisfies by returning the current state, any up-to-date 
status, and load information. 



The Configuration Manager uses the message service to request configuration informa- 
tion from the System Monitor. Although each server component could easily go to the 
database for configuration information, it has been decided to go through the monitor as 
to save db licensing costs. 

See below for more details on messaging protocols for the Server State Manager and the 
Configuration Manager. Also, refer to the low-level design document for details on the 

design of the eStream Messaging Service (EMS). 

Configuration Management: 

The configuration management utility is used by all server components to manage the 
server configurations. It provides the following fiinctionality: 

• Configuration for a server consists of a set of name - value tuples where the val- 
ues themselves can be a set of name- value tuple. 

• Servers can load the complete configuration from the database (indirectly). 



Omnishift Technologies, Inc. 



3 



Company Confidential 



eStream Server Component Framework Low Level Design 



• Servers can load the configuration for a given name. 

• Servers can load the configuration fi-om a flat file also. 

On the Server Manger interface, configuration will appear as a table containing name - 
value tuples. The table may be hierarchical to represent nested structures containing the 
values which can themselves be name values. An example of a simple name-value pair 
would be: 

port 8080 

An example of nested name values would be: 

Applications: 

word.exe windows2000sp3 
excel.exe win98sp4 

On a flat file the configurations will always be name-value pairs. To represent one level 
nested structure the format would be: 

Applications word.exe windows2000sp3 
Applications excel.exe win98sp4 

A common set of configurable parameters is defined for all server components. These 
configurations are maintained by the Server Component Framework in collaboration with 
the Configuration Manager. All configuration information is persistently stored within 
the database. The common configurations are used to initialize the server component 
after the component process has been launched. Refer to the configuration table below 
for more details on common configurations. Specialized server components can support 
additional configurations (non-common) depending on the server type. These 
configurations are read from the database and updated when a server component starts 
processing. They can also be updated dynamically while a server component is 
processing through the use of the UpdateConfig interface. 



The list of common configurations include: 



Information 


Supports 
Dynamic 
Config 


State 


Description 


ServerlD 


No 




Unique identifier for server components. This server 
identify is unique within a deployment. This 
ServerlD is not known to eStream clients. Its purpose 
is as a handle to uniquely identify server components. 


ServerType 


No 




Identifies the type of server component. One of the 
following applies: 

■ Primary Monitor 

■ Backup Monitor 
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■ Application Server 

■ SLiM Server 


DbUser 


No 




User name string required for database connectivity 
for this server ID 


DbPasswd 


No 




Database password associated with the DbUser 


Dsn 


No 




Data Source Name used to access the database. 


PortNum 


No 




PortNumber used for light-weight messaging listener 


MachinelD 


No 




Machine ID is used to get at important machine in- 
formation needed for all server components such as: 

■ IP address for the machine server component is 
hosted on 

■ Domain name for the machine 

■ Machines name 


AutoReStart 


Yes 


Any 
State 


Flag indicating that server component process can be 
restarted automatically without manual intervention 


TimeOut 


Yes 


Any 
State 


Specifies the timeout period for the listener. If the 
timeout period is reached. The component assumes 
that it has lost the connection. All Server components 
have a listener by which they receive instructions 
from the primary system monitor. Even the monitor 
has a listener that communicates with the Server 
Admin UI. 



Command Line Utilities: 



The Command Line Utilities component provides a consistent way to define and process 
command line arguments. To use this utility, the using component must define a table of 
arguments, which defines the valid set of arguments, whether or not they are required, 
and any default values. 

Arguments are specified on the command line as name/value pairs. The utility imple- 
ments the following command line syntax to support the name/value pairs. The argument 
syntax is defined as follows: 



<name>=<value> 



name 


Name is an alpha-numeric identifier. The Name can be of arbitrary length as 
supported by the system however shorter names are recommended. Names are 
case sensitive 


value 


Any alpha-numeric value. Punctuation characters may also be used. Values are 
case sensitive 



There can be no spaces between the <name>, "=", and the <value> elements. The exis- 
tence of one or more spaces or tabs delineates separation between arguments on the 
command line. 
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Example: server.exe sid=267 dns=oracle user=michaelb passwd=mypasswd 

□ If a named argument is specified more than once on the command line, subse- 
quent arguments will cause a diagnostic to be issued and the argument will be ig- 
nored. 

□ This utility allows the user to specify default values for arguments. If a default 
value is defined then the argument will be processed with its default in the event 
that the argument is not specified on the command line. 

□ This utility allows the user to tag specific arguments as required. If the required 
argument is not specified on the command line this utility will raise a diagnostic 
for the required argument. Not specifying a required argument will cause a fatal 
error. 



The following options are supported: 



sid 


Server Component Identifier. Each server component within a deploy- 
ment is uniquely identified via the sid. The sid is a handle into the data- 
base for accessing information unique to a specific server component. 


dsn 


Data Source Name. A data source name is necessary to establish an 
ODBC connection. Data Source Names are generated by an ODBC ad- 
ministrative tool 


dbuser 


User name. For database access security, all components need to connect 
as a specific user. 


dbpasswd 


password associate with the dbuser 



Logging Utilities: 

All servers and clients in eStream 1 .0 need to log the error and access data. Logging en- 
ables component debugging and auditing support. 

EStiream Framework should provide logging with the following features: 

• Each component will have an error and optioanally an access log file. The names 
of these files would be <component>_error.log and <component>_access.log. 

• The files will be located in the <eStreaml .0 Root Dir>\logs directory. 

• The error log files will have messages with the following priorities: 

o 4-Low : A warning which can be ignored. 

o 3-Medium: A warning which needs to be looked into. 

o 2-High: Recoverable Error in the component. 

o 1 -Critical: Fatal Error. Needs admin assistance. 

• Logging level should be configurable. The following levels are to be supported. 

o 0: Only errors will be logged. This will be the default level, 
o 1 : Errors and Warnings to be logged. 

o 2: Errors, Warnings and Debugging information to be logged, 
o 3: Errors, Warnings and advanced Debugging (like memory dumps, tcp 
stack dumps etc) to be logged. 
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• Log Wrapping to be supported. The log files will wrap at a predefined size. On 
wrapping the following actions will occur: 

o Any existing <logfile>.bak will be deleted firom the system, 
o The current <logfile> will be backed to <logfile>.bak 
o The component will continue logging to the <logfile>. 

For each eStream cUent and server component logging the log files (component_error.log 
and component_access.log) should be written in eStreaml.ORootUogs directory. The 
formats for the log files will be as follows: 



Error Log: 



[HEADER] 

[TimeStamp] [Thread ID] [Priority] [Message] 
[FOOTER] 

An example of this log format would be: 



Omnishift eStream Application Server 
Server Started. 

StartTime: 14/Aug/2000:16:31:19 -0700 
IP Address: 1.1.1.1 
Logging Level: 3 



[14 /Aug/2000: 16: 31: 19 -0700] 0 2-High Cannot connect to the database. 
Invalid Username/Password. 

[14/Aug/2000:16:31:19 -0700] 1 1-Critical Cannot start the HTTP listener 
at port 80. 

[14/Aug/2000:16:31:19 -0700] 0 1-Critical Shutting down the server. 

stc********************************************* 

Omnishift eStream Application Server 
Server Stopped. 

StOpTime: 14/Aug/2000: 16:35: 19 -0700 
IP Address : 1.1.1.1 
Logging Level: 3 



Format of Access Log Message: 
[HEADER] 

[TimeStamp] [Thread ID] [Message] 
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[FOOTER] 

Data type definitions 

Server State: 



The server components can be in any one of the following states: 



State 


Description 


STOPPED 


If a server is in the STOPPED state then the component 

process is not running. 


IDLE 


Server component is up and running. The server has been 
initialized with the common configuration and the messag- 
ing system has been enabled. 

The listener is actively waiting on the System Monitor for 
transition requests. 

The server component is not processing any work specific to 

this servers specialization. 


PROCESSING 


Server component is actively taking requests and processing 
work specific to its specialization, ie. serving access tokens, 
and application file requests. 


ERROR 


An error has occurred in the system. Unless the server 
component is configured with AutoReStart and ERROR 
state must be manually cleared by the server-side adminis- 
trator. 



Server State Transitions: 



Changes in server component state are initiated either by the System Monitor or directly 
by the server-side administrator for the system monitor. The exception to this is when an 
error condition is raised by a server component. In this case, the component will initiate 
the state change itself The following state transitions are supported: 



Action 


Description 


START_SERVER 


Server is expected to be in the STOPPED state. 
If a server component is configured to support AutoReStart 
then the ERROR state is also a valid state from which to initi- 
ate this action. 


STOP_SERVER 


Causes the server to exit its process. The server can be stopped 
fi-om any state. 


STARTPROCESSING 


Causes the server to change fi-om the IDLE state to the 
PROCESSING state. 


STOP PROCESSING 


Causes the server to change from processing to IDLE state. 


UPDATECONFIG 


Request that the server read its configuration from the configu- 
ration manager and change its configuration. 
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RAISE_ERROR 


Request that the server go to ERROR state. This causes an er- 
ror handler to be called. If the error is fatal it will cause im- 
mediate termination of the server process. 


Finite State Table: 



FSMTableEntry ServerStateMgr::FSMTable[] = 



{ START, {{START_SERVER,STOPPED, START_SERVER,NULL}, 

{START_PROCESSING, STOPPED ,START_PROCESSING, NULL}, 
{NULL REQUEST, NULL_STATE, NULL_REQUEST, NULL} } }, 

{STOPPED, {{START SERVER, IDLE, NULL_REQUEST, &StartServer}, 

{START PROCESSING, IDLE, START PROCESSING, fcStartServer}, 
{RAISE_ERROR, ERROR, NULL REQUEST, &HandleError}, 
{NULL_REQUEST, NULL_STATE, NULL_REQUEST, NULL} } }, 

{IDLE,{{START_PROCESSING, PROCESSING, NULL_REQUEST, 
&StartProcessing}, 

{STOP SERVER, STOPPED, NULL_REQUEST, &StopServer}, 
{RAISE_ERROR, ERROR, NULL_REQUEST, &HandleError} , 
{UPDATE_CONFIG, IDLE, NULL_REQUEST, &UpdateConfig}, 
{NULL_REQUEST, NULL_STATE, NULL_REQUEST, NULL} } }, 

{PROCESSING, {{STOP_PROCESSING, IDLE, NULL_REQUEST, 

&StopProcessing}, 

{UPDATE_CONFIG, PROCESSING, NULL_REQUEST, 

&UpdateConfig}, 

{STOP_SERVER, IDLE, STOP_SERVER, &StopProcessing} , 
{RAISE_ERROR, ERROR, NULL_REQUEST, &HandleError}, 
{NULL_REQUEST, NULL_STATE, NULL_REQUEST, NULL} } }, 

{ ERROR, {{STOP_SERVER, STOPPED,NULL_REQUEST, NULL}, 

{NULL_REQUEST, NULL_STATE, NULL_REQUEST, NULL} } } 

{NULL_STATE,{{NULL_REQUEST, NULL_STATE, NULL_REQUEST, 
NULL} } } 

± 

Messaging Service Protocol: 

A light-weight messaging protocol is needed to facilitate communication between server 
components. The primary purpose of the messaging protocol is to communicate transi- 
tion requests to the server components. In response, server components communicate 
state, status, and load information back to the System Monitor. 
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The messaging protocol supports two primary message types. 1) Requests for the System 
Monitor to perform on other servers. 2) Requests to the server components themselves. 
These message types are distinguished through the protocol as described below. If the 
receiver ID and the target ID are identical then the request is for the receiver. If the target 
is different than the receiver, the message is for the System Monitor to enact a request on 
the target component. 

All requests are required to be acknowledged. Without an acknowledgement the message 
is considered un-received. 

I OpCode I senderlD j receiverlD | targetID | Data | 



The following table describes the protocol used by the Server State Manager in its com- 
munication with the System Monitor. 



OpCode 


Description 


Data 


0x01 


Request for current state 


None 


0x02 


Acknowledgment 


■ Current state 

■ Load info 

■ Status info 


0x03 


Stop Server request. Acknowledged with 0x02 
message 


None 


0x04 


Start Server request. Only valid for System 
Monitor. Acknowledged with 0x02 


None 


0x05 


Start Processing Request. Acknowledged with 

0x02 


None 


0x06 


Stop Processing Request. Acknowledged with 

0x02 


None 


0x07 


Update Configuration Request. This is a re- 
quest for a server component to request its spe- 
cialized configuration information from the 
System Monitor and update itself Acknowl- 
edged with 0x02. 


None 


The messaging protocol used by the configuration manager is described below: 


OpCode 


Description 


Data 


0x01 


Request a complete reload of Configuration 




0x02 


Request a reload of specific configuration 
items. 


■ Number of items 
being requested. 

■ array of names of 
configuration items. 


0x03 


Acknowledgement of configuration reload re- 
quest. 


■ Number of tuples 
being returned 

■ Flat representation 
of configuration tu- 
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I pies 



Interface definitions 



Server State Manager: 



class ServerStateMgr 

{ 



pnvate: 



ServerState CurrentState; 
static FSMTableEntry FSMTable[]; 



public: 



ServerStateMgr(void); 
~ServerStateMgr(void); 

ServerState SetState(ServerState); 

ServerState GetState(void); 

ServerState ProcessRequest(ServerRequest); 



Description: Sets the current state of the server component. 

1 . Log the state change request 

2. Update the state field within the server component in memory data struc- 
tures. 

3 . Send message to requester informing them of the successful state change. 
Note: SetState does not update the database directly as in the orginal design. 
The database is updated by the System Monitor once it has received an ac- 
knowledgement. A state transition is not complete until SetState returns suc- 
cessfully and the Monitor has update the database. 

Input: state value to set current state to. 

Output: current state after the new value has been set. If an error occurs will 

go to error state. 

Errors: 

1 . Invalid state argument 

2. Failure to either connect or commit state change to the database. 



Description: returns the current state. This function does not read from the 
database to get the current state. The assumption is that if the server compo- 
nent is up and running and that it maintains a valid state. 
Input: none. 

Output: returns the current state. 

Errors: None. Will always return a valid state. 



ProcessRequest I Description: request to the Server State Manager to change server 
state. This routine implements the guts of the state machine. 
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1 . Get the current state, and transition request 

2. Index into the FSM table and continue to transition from state to 
state until the transition request is satisfied. 

3 . Each state transition calls the specialized transition routines for 
each component. 

4. Call to SetState to complete each state transition. 

5 . In the case of an error the state machine will process a 
RAISE ERROR request which will call the specialized Han- 
dleError and transition to the ERROR state. 

Input: server transition request. Refer to table of valid requests de- 
fined above. 

Output: current state after the request has been completed. 
Errors: ^ 



Server Component Framework: 

class ServerComponent: ServerStateMgr { // abstract base class 
private: 

Errorlnfo* Error; // maintains error if error was detected 
ServerConfig* Config; // holds common configuration 
Connection* Listener; // messaging ufility 

public: 

virtual int StartServer(void); // may be specialized by a server component 

virtual int StopServer(void); // may be specialized 

virtual int StartProcessing(void) = 0; // must be specialized 

virtual int StopProcessing(void) = 0; // must be specialized 

virtual int UpdateConfig(void) = 0; // must be specialized 

virtual int HandleError(void) = 0; // must be specialized 

void Run(Request); 
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StartServer 



Description: Called by the Server State Manager when a server compo- 
nent is to be started. The StartServer routine is provided as part of the 
SeverComponent class. It performs the following: 

1 . Send request to System Monitor to request an update of common con- 
figuration information. 

2. Apply the configuration information to the server component. 

3 . Construct a listener connection object and start the message service. 

4. Return success or failure. 
Note: 

■ This routine must return immediately to the main thread. Otherwise 
the Server State Manager will be blocked. 

■ Successfiil return from the StartServer routine will put the server 
into the IDLE state. 

Input: None. 

Output: Value of 0 if successful else error condition 

Errors: May return negative error condition 



StopServer 



Description: Called by the Server State Manager. 



1 . Perform any necessary cleanup. 

2. Send last acknowledgment confirming shutdown to requester 

3 . Shut down the messaging system and the listener. 

4. exit process 

Note: The monitor will update the database and perform logging. 
Input: None. 

Output: Value of 0 if successful else error condition 



Errors: May return negative error value 
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Description: Called by the Server State Manager. This routine must 



be defined by each specialized server component. This routine is 
used to provide all functionality unique to different types of servers. 
1 . Spawn a primary processing thread (also known as the boss 
thread). 

a. Read server specific configurations unique to this type of 
server component from the System Monitor 

b. Spawn worker threads. Depending on the server type this 
routine does the heavy lifting to either process access to- 
kens and renewals in the case of SLiM server, or process 
file requests for application servers, or manage and moni- 
tor the server components in the case of the System Moni- 
tor. 

Note: 

■ This routine must return immediately so that the Server State 
Manager can continue to operate in the main thread. 

■ This routine may make use of the Server Configuration Manager 
for obtaining specialized configuration information 

Input: None 

Output: Value of 0 if successful else error condition. 



Errors: TBD 



Description: Called by the Server State Manager. This routine must 



be defined by the specialized server component type. 
1 . Reverse all actions performed by the StartProcessing routine. All 

worker threads should be joined or pooled in waiting state. 
Successful return firom this routine will put the server component into 
the IDLE state. 
Input: None. 

Output: Value of 0 if successfial else error condition. 



Errors: TBD 



Description: Called by the Server State Manager. This routine must 
be defined by the specific server component type. The purpose of this 
routine is apply dynamic configurations or update specialized configu- 
rations that are unique to this server component. 
<may require adding a new state to separate dynamic and static con- 
figurations> 
Input: None. 

Output: Value of 0 if successful else error condition. 



Errors: TBD 



Description: Component defined error handling routine to handle errors 
such as timeouts, etc. 

This routine will need to handle a number of error cases as are possible 
by the specialized component. The error information is maintained with 



Omnishift Technologies, Inc. 



Company Confidential 



eStream Server Component Framework Low Level Design 





the ServerComponent class. 




Input: None. 

Output: Integer value designating a handled error or failure. It the error 




cannot be handled then it is fatal. 




Errors: TBD 1 



Description: This routine implements the main processing loop for the server 
component and runs in the main thread. This routine drives the server component 
by initiating state requests from the System Monitor. 
Note: The Server State Manager always runs in the main thread. 

1 . Call ProcessRequest to transition the server component into the mitially re- 
quested state. 

2. Enter main processing loop 

a. Check for requests from the message service. 

b Call ProcessRequest to service the request. 

c. Send acknowledgement for the request to the message service. Ac- 
knowledgement includes new state, load info, and status. 
Input: Initial Transition Request 
Output: None. This routine should never return 



Errors: None. 



Server Component Main Loop: 

The following main loop is common to all server components: 



void ServerComponent: :Run(ServerRequest Request) 

ProcessRequest(Request) ; 
while (1) 

Request = Listener->GetRequest(); 
ProcessRequest(Request); 

Listener->AckRequest(Request, GetState, GetLoad, GetStatus); 

} 



#include "ServerArgs.h" 
#include "Server .h" 



int main(int argc, char* argv[]) { 
Args = new ArgList(); 
Args->ProcessArgList(argv, argc); 
Server = new ServerComponent(GetValue(SID), 
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~" GetValue(DNS), 

GetValue(DBUSER), 
GetValue(PASSWD)); 
Server->Run(START_PROCESSING); 



Command Line Utilities: 



class NameValuePair 

{ 

private: 

char* Name; 

char* Value; 

pubUc: 

NameValuePairO; 
-NameValuePairO; 
char* GetValue(void); 
char* GetName(void); 
char* SetName(char*); 
char*SetValue(char*); 

Ji 



typedef int (*pFunc)(NameValuePair*); 

struct ArgTbl Entry 
{ 

char* Name; 
bool Required; 
char* DefaultValue; 
pFunc ProcessFunction; 



ArgTblEntry const ServerArgsTbl[] 
{"sid", true, 
{"dsn", true, 
{"dbuser", true, 
{"dbpasswd", true, 
{0, 0' 



1^ 



&ProcessSld}, 
&ProcessDsn}, 
&ProcessDbUser}, 



0} 



typedef vector<NameValuePair*> ArgVector; 



class ArgList 
{ 

private: 
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ArgVector 

const ArgTblEntry* 


ArgVec; 
ArgTbl; 


private: 

NameValuePair* 

char* 

char* 

int 

int 


ParseArg(char* Arg); 
ParseName(char* Arg); 
ParseValue(char* Arg); 
ProcessArg(NameValuePair*); 
FinalizeArgs(void); 


public: 

int 


ArgList(const ArgTblEntry*); 
ProcessArgList(char* argv[], int argc); 



Omnishift Technologies, Inc. 



17 



Company Confidential 



eStream Server Component Framework Low Level Design 



ProcessArgList 


DescriDtion: Process the entire argument hst. 
In a loop for each argument argv[] . . . 

1 . Call ParseArg passing in argv[] . 

2. ParseArg passes the result to ProcessArg 

T A ^♦■^^occ-inrr thp pTitirp arffumeut list and exiting the loop call 

3. /Viter processing ine ciiiiic cu guiuv-ui iioi. o r 

FinalizeArgs 

Input: argv and argc as passed into mainQ entry point 
Output: integer value designating success or failure 


ParseArg 


Description: Takes a char* argument and venfies that it follows that 
name/value syntax defined as <name>— <value> 
Input: Next char* argument on the list 

Output: NameValuePair. NULL will be returned in the event of a syntax 

error 


ProcessArg 


Description: This routine performs the semantic analysis of an argument. 

1 . Look up name in the ArgTbl 

2. Verify that the value is vahd 

3 . Add the name value pair to a list of processed arguments called ArgVec 
list 

4 If this name value pair already exists in the list then issue a diagnostic. 
5 . Call the supplied processing function for this argument as specified m the 

ArgTbl 
Input: NameValuePair 

Oi^t: Merger value designating success or failure (0 for success, positive 
integer for other errors) 


ParseName 


_ . \7„^A, tViQt tv><» Kinmp nart nf the areumcnt conforms to being 
Description: Venty tnat tne name pan ui uic aigmiiv^iii o 

alpha-numeric 

Input: char* Name part of argument 
Output: char* Name else NULL 

Error: None —7-^ 


ParseValue 


' J.- -i/„_;a, +v.ot tv>e» VqIiip nnrt nf the areumcnt coniorms to being 

Description: Verity tnat tne vaiue pan ui mc oigum^/iit 0 

alpha-numeric and/or punctuation characters 

Input: char* Value part of argument 

Output: char* Value else NULL 

Error: None — - — 


FinalizeArgs 


Description: Post process the argument list. 1 he purpose of this louliuc 1. tu 
validate that all required arguments have been defined on the command line. 
Also processes and adds default arguments to the ArgVec. 
Input: None 

Output: Success or Failure 

Error: 



Configuration Manager: 
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class Tuple { 

string name; 
Value value; 

}; 

class Value { 

int type; 

); 

class StringValue: public Value { 
string value; 

}; 

class TupleValue: public Value { 
vector <tuple> tupleArray; 

}; 

typedef vector < tuple > Config Array; 

class ServerConfig { 
private: 

ConfigArray Array; 

^^^^^^ ServerConfig(serverId, dns, dbuser, dbpasswd); // Initialize from 
ServerConfig(serverId, string filename); // To initialize from a fil 

ConfigArray* GetConfigArray(int serverld); 
Tuple* FindConfig(stringName); 
int Reload(void); 

Tuple* GetConfig(int serverld ,string Name); 



ServerConfig 


Description: Constructor for Configuration Manager. 

1 . Initializes configuration manager. 

2 Opens the database and gets configuration array 

Input: Server Id, Data Source Name, Database User name, and database 

users password. 

Output: None 

Errors: 


ServerConfig 


Description: Constructor for Configuration Manager. 

1 . Initializes Configuration Manager. 

2. Opens configuration file and reads configuration array. 
Input: filename of flat-file configuration. 

Output: None 

1 Errors: 
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GetConfigArray 


Description: Retiims the entire configuration for a given server id 
This routine always retrieves its information either from the flat tile 
or the database. . cr e 
Input: Serverld specifying which server to retneve configuration for 
Output: Retiims a vector holding the configuration or NULL 
Errors: 


GetConfig 




Description: Returns the configuration for the specitied name. This rou- 
tine always retrieves its information either from the flat file or tiie database. 
Input: Serverld specifying the server to retrieve configuration for and 
Name of configuration item. xttttt -f 
Output: Configuration Tuple. A Tuple may be a nested Tuple. NULL it 
an error is encountered. 

Errors: — 


FindConfig 


-^^^^^T-T^j^r^^ This routine does 
not go to the database or flat-file to get its value. Rather it finds the value 
in the ConfigArray maintained by the Configuration Manager. 
Input: Name of the configuration item. 

Output: Configuration Tuple. NULL if an error is encountered or the Tu- 
ple does not exist in the current configuration. 

Errors: 


Reload 


Description: Reloads the entire configuration from the database or flat- 
file This routine may reload its configuration indirecfly through the use ot 
the System Monitor. In this case it will make a message request to the 
monitor and listen for the configuration results. 

orput^iTteger specifying success or failure. Zero will be retiimed in the 
case of Success. A negative value in case of error. 

Errors: 



Logging Utilities: 



class LogManager 

{ 

private: 

char* FileName; 
int MaxFileSize; 

char* ResourceFile; // message catalog file 
char* GetMessage(MsgNum, MsgStr) 

pubUc: 

LogManager(ServerId,Size=l 0); 
LogMessage(MsgStr); 

LogMessage(ThreadId, MsgNum, MsgSti, ■■■); 
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LogMessage 


Description: Write message out to log file. There are two forms of 
LogMessage. The first will write out a message buffer as is (unformatted) 
bypassing the resource file. n-, 
The second form will format the message. Both forms of LogMessage 
always pre-append a time stamp. 

1 . Lookup message number in the resource file and get message stnng 

2. format the log message using time stamp, thread id, etc. 
3 write out message into the log file. 

Input: Thread Id, Message Number, Message String, and vanable num- 
ber of arguments. 
Output: None. 

Error: 




Description: Routine returns a message stnng firom the resource file tor 

the message number specified. 

Input: Message number, C Locale text string. 

Output: Message string. Either way. Get Message will always pass a 
return a valid message string by either returning the string from the re- 
source file or by passing back the MsgStr passed in. 
Error: If an error occurs trying to get a message from the resource file, a 
message will be logged to the error log. 1 



class ErrorLog: protected LogManager 

{ 

private: 

LogLevel ErrorLogLevel; 

public: 

ErrorLog(ServerId, LogLevel=0, Size-10); 
LogError(ThreadId, ErrorNum, ErrorMsgStr, ...); 
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LoeError I Description: Writes output to error log file. 

'l Check that the message level against the current ErrorLogLevel. 
2. Format the message and call the long form of LogMessage to wnte the 

buffer out to the file. 
Input: 

1 Threadid: Thread identifier to help with the debugging process. 
2. ErrorNum: Error number used to uniquely identify an error message in 

the resource file. . 
3 ErrorMsgStr: Message string which includes stdio like stnng formatting. 
4. . . . : variable list of arguments to be inserted into the message string per 

the format. 
Output: None. 

Error: 



Testing design 

Each of the components that make up the Server Component Framework 
be tested independently of the other components. Each component will h 
try point defined within a testing .exe to accomplish the Unit testing phas 



Testing of the component framework will be done in phases. Each of the phases ii 
scribed below along with its dependencies. 



Phase 1: Unit testing 

Test basic components that make up the frame- 
work. Each components functionality, restric- 
tions, and boundary conditions will be tested. 

Will allow testing common configurations for a 
single server component. This round of unit test- 
ing will test the integrated component utilities 
and framework. 


Dependencies: 

1 . ServerComponent class 

2. ServerStateMgr class 

3. ArgList class 

4. Logging Utilities 

5. Configuration Manager (flat-file) 


Phase 2: Unit testing (full functionality) 
Test full functionality including messaging inter- 
faces and database connectivity. 


Dependencies: 

1 . Phase 1 

2. Database connectivity 

3. Messaging Service 


Phase 3: Integration Testing 


Dependencies: 

1. Phase 2 

2. System Monitor (including 
backup) 

3. SLiM Server, App Server, Web- 
Server 


Phase 4: Stress Testing 

See section on stress testing for details 


Dependencies: 

1. Phase 3 
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TTnit testing plans 

Command Line Utilities 

The Command line utilities will be tested in a stand-alone module called cmdline.exe. It 

will su] 



ipport the command line arguments defined in this document. 



Configuration Manager 

The configuration module is a stand-alone module which will be tested using a config- 

S .exe executable. Tbe executable will ~ all of the inter a^^^^^^^^ 

The configtest.exe executable should be testable m the DB and the non-DB mode. 

Logging Utilities 

The logging utility will be buih as a DLL (otlog.dll). We will provide a binary otlog- 
test.exe which will exercise each of the interfaces mentioned above. 

Server State Manager 

The Server State Manager and the Server Component Framework will be tested ind^^^^ 
pendently of specialized components. The routines that require f ^^f^^^^^^^^.f J^^*" . 
Processing, StopProcessing, HandleError and UpdateConfig) will be provided to sim- 

ply return successfally. 

Stress testing plans 

Stress testing will require having at least the System Monitor functionality implemented 

since it is used to drive the server components. 

1 . Test to repeatedly start, stop, reconfigure the server component. 

2. Test to crash machines with server components to validate. 

a. data persistence. 

b. detection capabilities and response. 

c. auto restart. 

3. Test to kill individual server component processes. 

a. data persistence. 

b. detection capabilities and response. 

c. auto restart. 

4. Test lost database connectivity 

5. Test lost of messaging capabilities 

a. repeatedly losing and re-establishing messaging connectivity 
6 Test error recovery under adverse conditions. 
1. Test recovery from running out of memory, thread resources. 

8. Test recovery firom threads dying. 

9. etc. 

..rv^ 1 1 ■ 9-3 Company Confidential 
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Coverage testing plans 

1 Goal: 100% path flow coverage. Only exceptions for known error conditions that 
cannot be practically reached (e.g. thread synchronization, etc.) 

Cross-component testing plans 

The following pair- wise testing will be performed: 

1 . framework/database (phase 2) 

2. framework/messaging (phase 2) • ^ / -u a\ 

3 . framework (System Monitor) /framework (backup Monitor) (phase 3) 

4. framework/Web Server (phase 3) . . , 

5. framework (System Monitor) /framework (Other Servers) (phase 3) 

Upgrading/Supportability/Deployment design 

1 . Each error condition will be documented with explanations and practical work- 

2. Comptient framework will support enhanced debug option to dump additional de- 
bugging information to special log files. 

Open Issues 
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eStream Set Format Low Level Design 



Sanjay Pujare and David Lin 
Version 0. 1 
5 September 2000 

Functionality 

The eStream Set is a data set associated with an application suitable for streaming over 
the network. The eStream Set is generated by the eStream Builder program. This pro- 
gram converts locally installable applications into the eStream Set. This document de- 
scribes the format of the eStream Set. 

Data type definitions 

The format of the eStream Set consists of 4 sections: header, Root Version Table (RVT), 
Size Offset File Table (SOFT), and Concatenation Application File (CAF) sections. 
Fields greater than a single byte is stored in little-endian format. 

1. Header section 

o MagicNumber [4 bytes] : Magic number identifying the file content with 

the eStream Set 

o ESSVersion [4 bytes] : Version number of the eStream Set format, 
o AppID [16 bytes]: A unique application ID for this application. This 

field must match the AppID located in the AppInstallBlock. Guidgen is 

used to create this identifier. 

o RVToffset [4 bytes] : Byte offset into the start of the RVT section. 

o RVTsize [4 bytes] : Byte size of the RVT section. 

o SOFToffset [4 bytes] : Byte offset into the start of the SOFT section. 

o SOFTsize [4 bytes] : Byte size of the SOFT section. 

o CAFoffset [4 bytes] : Byte offset into the start of the CAF section. 

o CAFsize [4 bytes] : Byte size of the CAF section. 

o VendorNameLength [2 bytes] : Length of the vendor name. 

o VendorName [X bytes] : Name of the software vendor who created this 

apphcation. I.e. "Microsoft", 
o AppBaseNameLength [2 bytes] : Length of the application base name, 
o AppBaseName [X bytes] : Base name of the application. I.e. "Word 

2000". 

o MessageLength [2 bytes] : Length of the message text, 
o Message [X bytes] : Message text. 
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2. Root Version Table (RVT) section 

The Root version entries are ordered in a decreasing value according to their file 
numbers The Builder generates unique file numbers within each eStream Set m a 
monotonically increasing value. So larger root file number implies later versions 
of the same application. The latest root version is located at the top of the section 
to allow the eStream Server easy access to the data associated with the latest root 
version. 

o NumberEntries [4 bytes] : Number of patch versions contained in this 
eStream Set. The number indicates the number of entries in the Root Ver- 
sion Table (RVT). 

Root Version structure: (variable number of entries) 

o VersionNumber [4 bytes] : Version number of the root directory, 
o FileNumber [4 bytes] : File number of the root directory. 
o VersionName [32 bytes]: Application version name. I.e. "SP 1". 
o Metadata [28 bytes] : See eStream FS Directory for format of the meta- 
data. 

3. Size Offset File Table (SOFT) section 

The SOFT table contains information to locate specific files in the CAF section. 
The entries are ordered according to the file number starting firom 0 to Number- 
Files-1. 

o NumberFiles [4 bytes] : Number of entries in this section. 

SOFT entry structure: (variable number of entries) 

o Offset [4 bytes] : Byte offset into CAF of the start of this file, 
o Size [4 bytes]: Byte size of this file. The file is located firom address Off- 
set to Offset+Size. 

4. Concatenation Application File (CAF) section 

CAF is a concatenation of all file or directory data into a single data structure. 
Each piece of data can be a regular file, an AppInstallBlock, an eStream FS 
directory file, or an icon file. 

a. Regular Files 

o FileData [X bytes] : Content of a regular file 
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b. ApplnstallBlock (See ApplnstallBlock document for detail format) 

A simplified description of the ApplnstallBlock is listed here. For exact detail 
of the individual fields in the ApplnstallBlock, please see ApplnstallBlock 
Low-Level Design document. 

o Header section [X bytes] : Header for ApplnstallBlock containing infor- 
mation to identify this ApplnstallBlock. 

o Files section [X bytes] : Section containing file to be copied or spoofed. 

o AddVariable section [X bytes] : Section containing system variables to 
be added. . 

o RemoveVariable section [X bytes] : Section containing system variables 
to be removed. 

o Prefetch section [X bytes] : Section containing pointers to files to be pre- 
fetched to the client, 
o Profile section [X bytes] : Section containing profile data, (not used m 

eStream 1 .0) 

o Comment section [X bytes] : Section containing comments about Appln- 
stallBlock. 

o Code section [X bytes]: Section containing application-specific code 
needed to prepare local machine for stieaming this application 

o LicenseAgreement section [X bytes] : Section containing hcensing 
agreement message. 

c. EStream FS Directory 

An eStream FS Directory contains information about the subdirectories and 
files located within this directory. The information includes file number, 
names, and metadata associated with the files. 

o MagicNumber [4 bytes] : Magic number for eStream FS directory file. 

o FUeNumber [4 bytes] : File number for each file in this directory. 

o HashValue [2 bytes] : Hash value for quick filename value comparison to 

bypass the more expensive comparison of string names, 
o FileNamePointer [4 bytes] : Pointer to offset in this file where the long 

file name is located, 
o Metadata [32 bytes]: The metadata consists of file byte size (4 bytes), file 

creation time (8 bytes), file modified time (8 bytes), file size (4 bytes), file 

flags (4 bytes), eStieam flags (4 bytes). The bits of the file flags have the 

following meaning: 

■ Bit 0 : Read-only - Set if file is read-only 

■ Bit 1 : Hidden - Set if file is hidden from user 
The bits of the eStream flags have the following meaning: 

■ Bit 0: ForceUpgrade - Used only on root file. Set if client is 
forced to upgrade to this particular version if the current root ver- 
sion on the client is older. 
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■ Bit 1 : RequireAccessToken - Set if file require access token be- 
fore client can read it 
o LongFileNameSubsectionSize [4 bytes] : Size of the long file name sub- 
section. . ■ 1 f:i 
o LongFileName subsection [X bytes] : Subsection containing long tile 
names in null terminating string concatenated together. 

d. Icon files 

o IconFileData [X bytes] : Content of an icon file. 
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Format of the eStream Set 



Root Version Table section 



Size Offset File Table section 



Root Version Table (RVT[ 



I ^ Version i Root File ; Vereion Metadata -^i^ 



Size Offset File Table (SOFT) 



2 i Offset 



Patched File Data 



Offset 
Offset 



ApplnstallBlocl< 



eStream PS Directory 



Add Variable Section 

Profile Section 
Comment Section 

License Section 



Hash File Name Ptr 
Long File Names subsection 



Open Issues 



Where is the metadata associated with the Root directory bcated? Cur- 
rently root metadata is located in the root version table. All other files 
and directory metadata can be found in their parent directory. 
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eStream 1.0 High Level Design 



Version 0.3 
31 July 2000 



The following is roughly what's changed since the last version (0.2): 

□ The functional requirements and use cases have been removed. These will be 
documented in the eStream Requirements Document m future revs. 

□ The entire accounting hierarchy (what is a user and account, how are they 
grouped, at what level does billing take place) is undergomg revision, and has 
been removed from here for this version. 

□ Component descriptions should be more consistent now. , , , ^. 

□ The database of user and subscription information in the client block diagram has 
been removed. See the notes below. 

Known issues 

□ The mechanism for how a pathname on a client machine translates into a globally 
unique FilelD for any eStream server is unclear. This is a major design issue that 
crosses many components on both the client and the servers. 

□ The accounting hierarchy and its impact on this design are missing. 

□ If and how copy-on-write will work for writes to the Z file system is quite 

unknown. ... . • 

□ Which server manages user/account/group/subscription data is quite uncertain. 
Representing this by a data cylmder was wrong, and I removed this. However, all 
the interfaces specified below for m "ASP web server" are now just plain wrong, 
and the server team needs to suggest the appropriate changes to the HLD for the 
server topology. j ^ u 

□ The "Server Data Objects" section at the end of this document needs to be 
rewritten, in terms of interfaces that client and server components supply to 
support these data. 

Introduction 

This document describes the high level design for the eStream 1.0 product. The 

organization used is: 

□ Definitions , • n • 

□ Block diagrams for both the client and server portions, showing all major 

components 

□ Each component, generally broken down by 

o purpose 
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o functionality 

o global data managed, if any 

o interfaces for use by other components 

To understand the problem being solved in this design, see the "eStream Requirements 
Document" for information. 

Definitions 

account 

A billing entity consisting of a set of users and subscriptions 
user 

An entity authorized to use an account 
subscription 

An agreement between user and the ASP to use an application under terms of licensing. 
license 

Legal right to use an application at any given time. 
account admin 

A special kind of user who can add/delete other users from an account. 
server admin 

Administrator for all the eStream servers and database. 
AppID 

A unique representation of an application. There is one to one mapping of AppIDs to 

apps. 

FilelD 

Within an application, a unique representation of a particular file. 
access tolten 

This represents the right to run an eStreamed application. The client must acquire an 
access token before accessing any file (e.g., executing) in an eStreamed app. 
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application 

An application is the set of all files and directories, served by an eStreain sender, that 
make up a subscribed application. For example, any execu able file DLL icon file, or 
data file associated with an eStreamed version of FrameMaker is part of this application. 



application Installation 

This is the process of locally instaUing all bits necessary to execute an apphcation via 
eStream. Most files in the apphcation can be read or executed via the eStreamfile 
system ; some must be installed locally. Some configuration data must also be 
downloaded and processed to allow seamless execution of apps. 

app install block 

This is what needs to be downloaded and installed during ap plication installation . It 
might consist of: 

all configuration files that must be installed on the client machine 
all registry spoofing information required to run the app 
all file spoofing information required to run the app 
the names of all files and directories that make up the application 
initial prefetch data 

initial pages for critical application files 

It's quite possible that this app install block is actually an executable file or a DLL that 
performs all actions to make an apphcation ready to run, rather than simply a block of 



ASP ID block 

An ASP ID block consists of all the information about the applications available to a 
given user for a given ASP, on a given cUent machine. Since a user might be ong to 
multiple accounts for an ASP, this represents all subscribed apphcations for all accounts 



for that user. 



Such data might consist of: 



• user name 

• password 

. ASP contact info (IP address, URL, etc.) 

• list of subscribed apps 

o is the app installed on this machine? 
o serial number for app 

o ADRM server(s) to use for validation of this app 
o App server(s) to use to retiieve the app install block 
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o App server(s) to use to retrieve app file data 
. last time stamp when the ASP was checked for new subscnptions 

client certificate 

The client certificate for a dient machine is a digital signature used to identify it to 
eStream servers. We anticipate this to be used for requests that don^ require ati access 
token -- i.e., a valid license. For example, retrieving the app mstall block, or data tor 
eStream application files that don't require license vahdation. 

client machine 

This is a computer on which an eStreamed appHcation executes. It may host multiple 
registered users, and a single user can install the eStream client on multiple chent 
machines. 

eStream client 

This is the aggregate of all the software required on a client machine to subscribe to, 
install, and execute an eStreamed application. 

eStream file system 

The eStream file system (or EFS) is a distributed file system with prefetch and caching 
fianctionahty. All file data and metadata accessed through the EFS is subject to hcense 
validation before being available fi-om a server. 

license validation 

The act of validating a license means gaining an access token . Generally, before an 
eStream application can be run on a client machine, the validity of using this application 
by the current user must be checked. This check is done when certain files associated 
with the ap pUcation are accessed; an eStream server is contacted to perform this check 
and return an access token. 

subscription serial number 

Each application that a user is associated with a serial number. This identifies both the 
application and the user uniquely, and hence can be checked easily dunng license 

validation . 

Block diagram 

The following are simple block diagrams of the dient and server components. Some 
conventions: 
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□ A box represents a logical eStream component. A component may exist as a 
distinct process, or it may be grouped with other components into a common 

□ A^nTbetween components represents an interface call from one to ^^^^^ If A 
calls B, there's a arrow on the end of the line at B. If A and B call each other, 
there's an arrow on both ends of the line. 

Note that data stores are not represented in these diagrams; if a data ^tore i_s c^^^^^^^^^ 
managed, then there is a component that has interfaces to allow access to these data. 
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eStream Client Block Diagram 
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Component descriptions 

Client components 

The client components are all identified in the block diagram above. Very briefly, some 
points: 

1 A web browser on the client machine will be used for most user interface 
requests: subscribmg to applications, requesting subscription and payment 
information, and so forth. Configuration of the eStream client software will be 
done using a UI which may be different from a web browser. Some thoughts on 
this are listed below. 

2 The eStream cache manager is the heart of the client software, and is the 
component that actually requests file data from the servers. ^ . ^ ^ . . 

3 The license subscription manager has the task of tracking all valid subscriptions to 
applications from an ASP, and tracking which applications have, or need, a 
license validation to access files. 

4 The app install manager's task is to wait until it's told to mstall a newly 

■ subscribed application, and then do so. It also keeps frack of what needs to occur 
when uninstalling an appUcation. 

5 The client network interface simply takes requests from the rest of the client 
components, and forwards them on to the appropriate eStream servers. 

6 The eStream file system (EFS, aka the "Z" file system) is a standard kernel-mode 
network redirector. It presents the normal FS interface to the rest of the NT 
executive, and requests data from the eStream cache manager to satisfy requests 

7. Thf registry and file spoofers are kernel-mode drivers that monitor registry calls 
and file open requests, respectively. , , ■ ui 

8. The No Cluster component is a very simple kernel-mode driver that disables page 
clustering for reads. 



Installation Manager 



Purpose 

Installation of the eStream client software is not different than installing any other client 
software package such as Winzip or Office. The eStream client -^^a lation is separate 
from the installafion and configuration of eStream subscribed applications. Some of the 
possible pieces that eStream would need to be installed are listed below. 



1 . Device Drivers 

2. Apphcations Executables 

3. AppUcation Components 

4. Shared Components 

5. Registry entries 
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6. Shortcuts and Start menus 

7. Help Files 

8. Uninstaller 



Once the pieces of the apphcation to be mstalled are brought together then an install 
program must be constructed. 

Functionality 

The Installation Manager consists of the following sub-components 
Installshield 

1 How much of the application does the user wish to install? 

2 Is the users system capable of running the application? 

3 Where does the user wish to install the application? 

4. Does the user have enough space to install the applications? 

Installshield has a wizard that will set up a project. When the install s^^i^^^ program is 
effect. 

Install From the Web 

web site. 
Uninstaller 

Installshield will provide an uninstaller when it builds the install program. 
Registry Settings 

There are three ways that Installshield can patch the system registry. 

1 Run regsvr32.exe on self-registering .dll files. When the uninstaller is run it will 

use regsvr32.exe /u to un-register the .dll file. 
2. Patch the registry statically. 
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3. Patch the registry based on Installation Options from the install shield script 
program. 

Artwork 

The Installshield program for eStream will require a splash screen and possibly one or 
two other artwork components. 

eStream Client UI Module 

The eStream Client UI module is a client component, currently expected to be running in 
user space. 

Functionality 

The eStream Client UI module supports reporting eStream-specific error & informational 
messages to the client user and soliciting replies when appropriate. It allows the eStream 
client user to view and change the list of applications currently installed on the client 
system and the list of ASP accounts currently known to the client system. 

interfaces 

ReportMessage ToEStreamClientUser(IN message) 

Display specified message in EStream Client UI message window. 

QueryEStreamClientUser(IN message, OUT response) 

Display specified message in EStream Client UI message window and solicit yes/no 
response for return to caller. 

Installed Applications UI 

The Client UI interface allows the user to request that the list of the applications currently 
installed on the client be displayed. The Client UI Module gets this list by calling 
AIM/GetAppInstallList(). 

The Client UI interface allows the user to select an application from this list to be 
uninstalled. The Client UI Module calls AIM/UninstallApp() to accomplish this. 

The Client UI interface allows the user to enter the information necessary to get a new 
application installed. The CHent UI Module calls AIM/InstallApp() to accompUsh this. 

The Client UI interface allows the user to request that the list of applications currently 
installed on this client be exported to a file, in a form which would allow that list to be 
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imported on another client. The Client UI Module calls AIM/ExportInstalledApps() 
accomplish this. 

The Client UI interface allows the user to request that a list of applications that was 
exported by eStream running on another client be imported from a file & installed. 1 
Client UI Module calls AIM/ImportAndInstallApps() to accomplish this. 



Known ASPs UI 

The Client UI interface allows the user to request that the list of ASPs currently known to 
the client be displayed. The Client UI Module gets this list by calling ???. 

The Client UI interface allows the user to select and connect to an ASP in the list. The 
Client UI Module accomplishes this by ???. 

The Client UI interface allows the user to select an ASP from this list to be deleted. The 

Client UI Module calls ??? to accomplish this. 

The Client UI interface allows the user to enter the information necessary to record 
information about a new ASP account. The Client UI Module calls ??? to accomphsh 
this. 

The Client UI interface allows the user to request that the list of ASPs currently known to 
this client be exported to a file, in a form which would allow that list to be imported on 
another chent. The Client UI Module calls ??? to accomplish this. 

The Client UI interface allows the user to request that a list of ASPs that was exported by 
eStream running on another client be imported from a file & installed. The Client UI 
Module calls ??? to accomplish this. 



eStream Cache Manager 



Purpose 

The eStream Cache Manager (ECM) is a client component, currently expected to be 
running in user space. Its goal is to: 

□ Handle all file requests from the eStream file system, either by using previously 
cached contents or requesting the contents from a server. 

□ Intelligently use prefetching of file data to reduce latency of pages requested from 
the EFS. 

□ Work with the license subscription manager to insure that all applications have 
appropriately validated licenses before their files are accessed. 
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Functionality 

Th. vru handles the volatile & non-volatile eStream cache on the client machine. It 

data, which may periodically be uploaded to a server. 
Interfaces 

The ECM takes requests from the EPS driver, and makes requests to the client network 

and LSM modules. 

In the descriptions below practically every call could fail for a variety of reasons. The 
associated e^r handling paths are not shown at this level of the desrgn. 

Open/Create(IN Filename, IN FileOptions, OUT Handle) 

Called from the EFS. 

This does the following basic tasks: 

□ If the filename and mode options correspond to a FilelD f 

and legal to use (from if s cache), it can just return *^ f°'*^,f^„,,, 
a Otherwise it must ask the LSM for an access token for this file. (This request 
mSy remm an access token previously created for other files making up 

□ u" ch' antp^fetching and/or active cache loading activities desired for the 

new app. 

□ Request a file handle from an appropriate app server, via the dient network 
component. 

□ Return the handle to the caller 

Close(IN Handle) 

Called from the EFS. 
This basically: 

□ Informs the LSM that the file is being closed 

□ Unloads (or marks as victims) app's cached entries as desired 

ReadON Handle, IN ReadOffset. IN ReadLength, IN BufferPtr. OUT BytesRead) 
Called from the EFS. 
This does the following: 
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□ Update the profile data for this file 

□ Check the cache for the requested data 

□ If not there, request the appropriate pages from an app server, along with any page 
prefetches that are needed, and place the retrieved data in the cache 

□ Fill in the buffer and return the number of bytes written to this butter 

Write(IN Handle, IN Buffer, IN Offset, IN Length, OUT BytesWrltten) 

Called from the EFS. 

This will use some copy-on-write scheme. It may be as simple as locking the written 
structures in the cache. 

Global Data 

ActiveAppsData Structure 

Table of data with an entry for each application that is currently active on the client. 
Fields for each entry are listed below. 

• AppPrefix 

• AccessToken 

• ServerName 

• FilesOpen 

• LocalPathName 

FilelD Type 

A globally unique identifier defined for each file associated with an eStream application. 
AlfStreL-managed files have these identifiers to allow a common & unambiguous 
mihod of me referencing between clients & servers & to simplify switching the chent to 
an alternative server. 

ProfileData Structure 

Exact contents of this data structure will be defined in the low level design phase; at this 
point, assume predecessor/successor pairs w/counts. 

Volatile & Non-volatile Caching Structures 

Exact contents of these data structures will be defined in the low level design phase. 
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License Subscription Manager (LSM) 
Purpose 

The LSM tracks current subscription information and determines the need for Ucense 
^1 da^n I t mformed of subscription changes from the ^^^^^^I and . qu^^^^^^^^ 
ECM to validate accessibility to different applications, based on the hcense model for the 
subscription to that application. 

Functionality 

The LSM tracks the users subscriptions to different ASPs; it is part of the client 
Component downloaded on a client machine. The LSM starts rum.mg when the client 
component starts running, and is remains active until it stops. 

The LSM has a few major tasks: 

1. Keep track of what subscriptions the current user has available firom all ASPs 
9 Determine which application a given file is a part of 

3. Acqr I access token to validate a license for file requests that require one 
There are two ways that the LSM updates its list of known subscribed applications: 

1. It may be informed of new subscriptions, or of applications that -^"^f; 
by the cUent UI, as part of a browser plugin m conjunction ^^th ^ ASPs web site. 

2. It may asynchronously poll an ASPs ADRM servers to get updated lists of 

subscribed apps. 

When the users start running any of the subscribed eStream appUcations-i.e., when any 
Isteam'ed file is opened-lhe ECM quenes the LSM before servicing any requests. The 
LSM^cks to ee which subscribed application this file belongs to and, i necessary, 
getfthe a;propriate access tokens from ADRM servers along f^^^,, 
fpplication servers that can be used to run the apphcations; it ^^^^J^^/ l^f^!^^.^^^ 
obtained when the comiection to the ASP was made. At the same time, the LSM can 
decXto cache the access tokens and the identities of the appUcation servers and decide 
to serve them directly firom its cache. 

The ECM informs the LSM when files open and close, and determines fi-om this when 
Lnhcations strand end. The LSM keeps track of when access tokens are expiring and 
access tokens when appUcations are running and the current 

one is about to expire. 
Giobai Data 

The global data managed by the LSM includes 
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1 The ASP ID Blocks which are obtained when the user on the machine establishes 
■ a connection with an ASP from which the user has subscribed applications. 

2 The access tokens and the identities of the applications servers that are obtained 
from the ADRM servers when the user tries to run the applications. 

Interfaces 

The LSM exposes the following set of APIs to the client UI: 
SubscribeApp(IN ASPId, IN AppID, IN Licenselnfo) 

This routine in turn will call the App Install Mgr to install the application on the client 
machine. This will return a Boolean stating success or failure. 

UnsubscribeApp(IN ASPId, IN AppID) 

This routine will NOT implicitly uninstall the application. Applications must be explicitly 
uninstalled. This will return a Boolean stating success or failure. 

GetAppList(OUT SubscribedAppList) 

This routine will return a pointer to a list of subscribed applications on the client 
machine. 

The LSM exposes the following set of APIs to the ECM: 
CheckAccessON Path, OUT Root) 

The LSM establishes a correlation between the Path and the AppID by querying the App 
Install Mgr This routine in turn may contact the ADRM server for appropnate access 
Sen This will return a Boolean stating success or failure. At the same time Root will 
get set to the head of the path that identifies the application so that the file system can use 
the same access token for everything under "Root". 

BeginApp(IN AppID) 

To indicate the start of an application. Note: this may happen impUcitly during 

Check Access(). 

EndApp(IN AppID) 

To indicate the end of the application. Note: this may happen impUcitly during 

CheckAccess(). 

The LSM makes the following API calls. 
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1 . InstallApp(ASPId, AppID) to the App Install Mgr to install the subscribed 

2. 



. Sl;pM;a*,&Roo.).ofteAppIns.allM^togettheAppIdfiomftePaft. 



"Root" is explained above. 

tofomatiorftom Ms server when fte ^er fct establishes a conneCon .t. 

Application InstaU Manager (AIM) 

Purpose 

The AIM is the contact point for installation and uninstallation of appHcations on a dient 
macWne gets the requests from the LSM to install applications when the user 
Tubtribef ^^^^^^^^ and it gets requests from the Client UI to unmstall 

applications. 

Functionality 

' al and to upLe *e star, menu and o4er short cuts as needed. 

Global Data 

The Global Data managed by the AIM includes - 

1 . The AppInstallBlock obtained from the app server that is used to do the 

2. m ApX>Path co-relation that is required to check for access privileges. 
Interfaces 

The AIM exposes the following interfaces - 
lnstallApp(IN ASPId. IN AppID) 

To install the appUcation using a specific ASP server to get the AppInstallBlock. 
UninstaltAppON AppID) 
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To uninstall the appUcation from the cUent machine. 
GetAppld(IN Path, OUT Root) 

To return the AppID given the Path that is being used to open a file/directory on the 

eStream file system. 

GetApplnstallUst(OUTInstalledAppList) 

To get a list of the applications currently installed. 

The AIM makes calls to the registry and the file spoofers using the AddRegSpoofEntry, 
AddFileSpoofEntry, etc. APIs. 

eStream client network component 

This section deals with the components that communicate with the servers. 
Purpose 

Tho client network component is the common point of connecfion ^e^we^^^^^^^^ 

eStream dient components and the various eStream servers. Any client module that calls 

an interface of a server does so through the network component. 

This component is basically stupid. It knows the protocols needed for communicating 
^h he vLus sewers, and U can encode the requested messages via the- protocols^ 
bu it doesn't try to be smart with regard to failover, or authentication rejection or other 
elr conditions. The network component lets its caller deal with such matters. 

One design assumption here is that data is received from an eStream server only in 
Response to a request it has made of this server. In other words, all requests onginate 
with the chent, never from the server. 

Functionality 

The client network component communicates with the following servers for the types of 
requests listed. 

ADRM server 

1 . Validate a user for this ASP and get subscription information 

2. Validate a license for a subscribed app 

App server 

1 . Open a file/directory for a subscribed app 
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2. Various file requests on a previously opened file/directory 
Global Data 
Probably none (?). 
Interfaces 

ValidateUser(IN ADRMServer, IN AspAndUserData, OUT Subscription! nfo) 

This interface is called by the LSM; it is used both to validate a user and get updated 
subscription information for a given ASP. 

ValidateLicenseON ADRMServer, INAPPId, IN ClientCertificate, OUT AccessToken, 
OUT AppServerList) 

This is called by the LSM, to get an access token for an application before its file can be 



AppOpenFile(IN AppServer, IN AccessToken, IN FileDesignator, OUT Handle) 

This is called by the ECM, to for any eStream file. Note that this is also used to retrieve 
an AppInstallBlock, when requested via the AIM. Note: the FileDesignator is still 
undergoing design. 

AppReadFile(IN AppServer, IN AccessToken, IN Handle, IN OUT Buffer, IN Offset, IN 
Length, OUT BytesRead) 

This is called by the ECM. 

UploadAppProfileDataRequest(IN ADRMServer, IN ProfileData, OUT Success) 
It's unclear who calls this! 
eStream File System Driver 

The file system driver interfaces with the operating system's installable file system 
facilities, forwards file system requests that it cannot directly satisfy to the ECM, and 
uses the NT File Cache to optimize repeated accesses to the same data. This component 
will be very operafing system specific, while the interfaces it exposes to the cache 
manager will be (mostly) OS independent. The file system driver resides m kernel space 
and implements a portion of the entire eStream file system. Other components, such as 
the cache manager, the client network interface, and the app servers, implement the rest 
of the eStream file system. These other components are not necessanly kernel-mode 
resident. 



Omnishift Confidential Page 1 8 9/2/2009 



eStream 1 .0 High Level Design 



version 0.3 



Functionality 

The eStream file system driver (EFS) will send most requests firom the operating system 
to the cache manager. It will interface with the standard NT File Cache Manager to avoid 
sending redundant requests to the cache manager. And it must support functionality for 
the ECM to notify it when the data structures it has cached have become invalid. 



Global Data 

The only globally- visible data managed by the file system driver are various things that it 
may cache. This includes both file data pages as well as directory contents. These data 
are relevant to the ECM, because it may want to invalidate the contents of the caches if it 
finds a newer version of a data page or finds that (visible) directory contents have 
changed. 



Interfaces 

These are the logical interfaces that are exposed to the ECM. The EFS has standard 
system interfaces that are used by the NT Executive, but these are not listed here. 

lnvalidatePage(IN FileHandle, IN PageOffset) 

Invalidates the specified page for the specified file handle in the cache. 



lnvalidateDirectory(IN DirHandle) 

Invalidates the specified directory's contents in the cache. This may result in the eStream 
file system driver sending directory change notifications. 

ShutdownFileSystem(IN Force) 

Attempts to shut down the file system. If Force is true, the file system will be shut down 
regardless of whether any processes still have handles that are open on the file system. If 
Force is false, this routine will return an error if there are any open file handles. After the 
file system is shut down, any attempt to access the file system will result in errors rather 
than being forwarded on to the cache manager, until StartFileSystem is called. 



StartFileSystemQ 

Causes the eStream file system to begin accepting requests and forwarding them to 
eStream cache manager. 
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Virtual Memory Clustering Disabling Driver 
Purpose 

The VM clustering disabling driver (aka NoCluster) disables virtual memory clustering 
under Windows. While we don't fully understand all the implications, using this driver 
substantially reduces the average file system paging request size and can dramatically 
improve performance of eStream, especially on slower connections. 

Virtual memory clustering, as implemented in Windows NT/2000, is intended to improve 
performance when paging to and from physical disks. If possible, we would like to 
disable clustering only for those threads/processes that will be doing a significant amount 
of I/O to the eStream file system. 

Functionality 

The VM clustering disabling driver maintains a set of criteria for threads whose 
clustering should be disabled. It will make decisions about which threads should have 
clustering disabled without contacting any other components. 

Global Data 

The only data managed by this component are the criteria for selecting threads whose 
clustering should be disabled, hi the simplest implementation, this would be nothing, and 
the driver would disable clustering for all threads. Whatever tables it uses will be 
resident in the kernel, and the driver will be able to access them as needed without 
making calls to a user-mode component to read them. 

Interfaces 

The interfaces for this component are minimal. Clustering may be enabled or disabled, 
and the criteria for which threads to manipulate can be changed. 

StartDisablingClusteringQ 

This interface notifies the driver that it should begin disable virtual memory clustering, 
using the cun-ently specified criteria. 

StopDisablingClusteringO 

This interface notifies the driver that it should stop disabling clustering. Note that due to 
implementation problems, we do not support actual unloading of the driver, though it can 
be removed from the system by a reboot. 

ChangeClusteringCriteria(IN Criteria) 
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This interface allows the caller to cause the driver to change the criteria it uses to select 
threads it should manipulate. The criteria might be specified in the interface, or it might 
specify a file that the driver should read for the new criteria. 

QueryClusteringCriteria(OUT Criteria, OUT Active) 

This interface allows the caller to find out what criteria the VM clustering disabling 
driver is currently using, and whether or not it is currently active. 

File Spoofer 
Purpose 

The purpose of the file spoofer is to redirect file system accesses fi-om some non-eStream 
drive. This may be necessary in order to support applications running under eStream that 
are hard-wired to access files in a specific location. The file spoofer may also be used if 
we are interested in providing a version of some system file different from the one 
actually on the client machine. 

Functionality 

The file spoofer will intercept File Create calls for files that we are interested in spoofing 
and ensure that these creates are redirected to a file we specify. The redirection could be 
to a file on the Z file system, or to another, non-eStream'ed file. 

File open is a very common occurrence, so the file spoofer must operate quickly. The file 
spoofer should maintain in-kemel whatever data structures it needs to make a spoofing 
decision. 

Global Data 

The file spoofer must maintain a database indicating which files to spoor, which file to 
replace them with, and possibly which processes should be spoofed. All of this 
information must be kept in-kemel so spoofing decisions can be made quickly. This 
database may change depending on which eStiream apps are currently installed or 
running. 

Interfaces 

StartFileSpoofingQ 

Causes the file spoofer to begin file spoofing, using the current spoof database. 
StopFileSpoofingO 

Causes the file spoofer to stop spoofing, but does not change the spoof database. 
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Add FileSpoofEntry(IN Spoof Entry) 

Adds an entry to the spoof database. It is not necessary to stop and restart the spoof 

database to add an entry. 

RemoveFileSpoofEntry(INSpoofEnty) 

Removes an entry from the spoof database. It is not necessary to stop and restart the 
spoof database to remove an entry. 

QueryFileSpoofDatabase(OUTSpoofEntryUst) 

Queries the current contents of the spoof database. 

ReplaceFileSpoofDatabase(IN Spoof DB) 

Replaces the entire spoof database. It is not necessary to stop and restart the spoof 
database to perform this action, and it is considered atomic. 

1.4 Registry Spoofer 
Purpose 

The puipose of the registry spoofer is to provide to eStreamed and other applications 
registry entries for eStreamed apps, and to capture registry writes by eStreamed apps so 
they can be purged from the registry or shipped to other clients for configuration 
ubiquity. 

Functionality 

The registry spoofer must redirect registry reads and writes. Because registry accesses 
are quite common, the redirector should be able to service registry spoofs without 
forwarding the requests to a user-level process. 

Global Data 

The registry spoofer maintains a spoof database of which registry entries to spoof, and 
which processes to spoof them for. This database should be kept in-kemel so that the 
spoof decisions can be made quickly. The spoof database may change depending on 
what eStream applications are currently installed or running. 

Interfaces 

StartRegSpoofingQ 

Causes the registry spoofer to begin spoofing using the current database. 
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StopRegSpoofingO 

Causes the registry spoofer to stop spoofing. This does not change the registry spoof 

database. 

AddRegSpoofEntry(IN Spoof Entry) 

Adds an entry to the registry spoofing database. It is not necessary to stop spoofing in 
order to do this. 

RemoveRegSpoofEntry(IN Spoof Entry) 

Removes an entry from the registry spoofing database. It is not necessary to stop 
spoofing in order to do this. 

ReplaceRegSpoofDatabase(INSpoofDB) 

Replaces the entire registiy spoofing database. It is not necessary to stop spoofing in 
order to do this, and it is considered an atomic operation. 

QueryRegSpoofDatabase(OUTSpoofEntryList) 

Queries the contents of the registry spoof database. 

Server components 

The servers described below are logical servers. Note that a single sei-ver machine can 
serve all fiinctions for a small ASP; alternatively, farms of servers can be used to provide 
the functionality of a single logical server. 

NOTE: The distribution of servers and the functionality provided by them are 
somewhat uncertain to date. In particular, exactly who manages the actual accounting, 
user, and group data is undecided. The group producing the LLD for the eStream servers 
need to flesh this out. For now, this document assumes that the ADRM server ultimately 
manages these data, and supplies interfaces to callers to access these data. 

The servers described are: 

1 . An ADRM server handles user/account/subscription data management, as well as 
validating licenses. 

2. An ASP web server is a front-end for requests to add users, subscribe to 
applications, and do various user and application level queries. Generally this 
forwards these requests to an ADRM server. 

3. An application server handles requests to open and read eStream files. 

4. A profile data server will receive uploaded profile data from a client machine to 
enable better initial profile and prefetch maps in eStream sets. 
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ADRM Server 
Purpose 

The Account/Digital Rights Management (ADRM) server is responsible for: 

□ Managing data related to users, the groups they belong to, and the applications 
they are subscribed to 

□ Validating the Ucenses for applications executing on clients 
a Tracking all outstanding hcenses currently in use 

Functionality 

Client machines sends requests to the ADRM server to add or delete subscriptions, to 
receive an access token to execute an application, and to manage their account/group/user 

relationship. 

Access tokens have an expiration time, so the client must reacquire them at regular 
intervals. When an eStreamed application exits, the client informs the ADRM server to 
release the access token. Any outstanding access token not released or reacquired within 
the expiration time will be automatically released by the server. 

Interfaces 

AcquireAccessToken(IN Userlnformation, IN Appld, OUT AccessToken, OUT 
AppServerList) 

This is called by the eStream client to gain validate a license before executing an 
application. 

This is used to insure that a user has the right to use a particular app in a subscription 
from a specific account. The server returns an access token and a list of app servers from 
which the client can access the application file data. If the user doesn't have a valid 
license to use the requested application, a failure message is sent to the client. The server 
writes the start time of this application usage into the database for billing processing. 

RenewAccessToken(IN OldAccessToken, OUT New AccessToken, OUT AppServerList)) 

Note: This may just be replaced by AcquireAccessToken() 

This is called by the eStream client. 

The server receives a message from a client to renew its access token before the 
expiration of the token. The server returns a new access token and a list of app servers. 
This allows the server to redirect the client to a different app server in case it knows of 
changes to the list of available servers. Once the token is expired, the ADRM server 
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writes the end time of this usage information into the database and the client must 
reacquire the access token before files for this appUcation are available to it. 

ReleaseAccessToken(IN AccessToken) 

This is called by the eStream client. 

The client returns the token to the server when the eStream app terminates so other 
clients can acquire the token. The server writes the end time of this usage information 
into the database for billing processing. 

AddApplicationServer(IN AppServer, IN ApplicationList) 

This is called by an appUcation server. 

The ADRM server is informed of the availability of a new application server. The 
ADRM server adds this new app server to its list of app servers. 

RemoveApplicationServer(IN AppServer) 

This is called by an application server. 

The ADRM server is told of the removal of an app server. It must remove this app server 
fi-om its list of such servers to prevent any clients from using that server. 

AddApplicationServerApplications(IN AppServer, IN ApplicationList) 

This is called by an application server. 

The ADRM server is informed of the availability of a new application on a given app 
server. The ADRM server adds this new app to the list of applications that the server has 
available. 

RemoveApplicationServerApplications(IN AppServer, IN ApplicationList) 
This is called by an application server. 

The ADRM server is told of the removal of an application for an application server. 

GetTrafficHistory(IN TimeCriteria, OUT HistoryData) 

This is called by a server UI tool, or possibly by other ADRM servers. 

The administrator monitoring, reporting, and management tool UI program can query the 
ADRM server for load information. The server logs all client requests to acquire access 
token. This raw information can either be sent directly to the UI program or it can be 
preprocessed before sending to the UI program. 
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GetErrors(IN TimeCriteria, IN ErrorType, OUT ErrorList) 
This is called by a server UI. 

The admin UI program can query the ADRM server for any errors. The errors can be 
categorized by type of errors or errors that occur between certain time periods. A small 
sample of the possible ADRM server errors includes: client access token timeout, failure 
to read user information fi-om the account database, failvire to get the license information, 
failure to write usage information into the database, etc. . . 

GetlllegalAccesses(IN TimeCriteria, IN AccessType, OUT AccessList) 

This is called by a server UI. 

The admin UI program can query the ADRM server for any illegal accesses. The illegal 
accesses can be categorized by type and time period. A small sample of the possible 
ADRM server errors includes: failure attempts to access ADRM server with bad 
password repeatedly in a small time period, failure attempts to use a particular license, 
any access attempts from a non-typical IP address ranges for a particular account, etc... 

GetApplD(IN AppName, OUT AppID) 

This is called by a server UI. 

Returns a unique identifier associated a particular application 
SetApp(ID IN AppName, IN AppID) 
This is called by a server UI. 

Stores a unique identifier associated a particular application 

Application Server 
Purpose 

The application server is there to handle read requests for files accessed by eStream 
clients. Any file accessed on a client through the EPS can have this read request passed 
to an app server. 

Functionality 

This will be the hardest working eStream server. It will respond to both synchronous 
(demand fetching) and asynchronous (prefetching) page requests from many different 
clients, for many different types of applications and files within those applications. 
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Interfaces 

GetFilelnfo(IN AccessToken, IN FilelD, OUTFilelnfo) 

This is called from an eStream client. Given any file within an eStream application, 
return metadata about it. The access token is provided for validation. 

ReadFileON AccessToken, IN FilelD, IN Length, IN Offset, OUT Buffer, OUT BytesRead) 

This interface is called by an eStream client, and will allow the client to access any 
eStreamed application file and AppInstallBlocks. How the FilelD for an ApphistallBlock 

is achieved is unclear at present. 

OpenFileO / GetFilelDQ 

Note: This is a placeholder for an API that may be needed. This depends a lot on the 
eventual communications between client and server for associating a file pathname with a 
FilelD. 

ASP web server 
Purpose 

This describes, of course, only those interfaces on an ASP web server that relate to 
handling eStreamed applications. 

Logically, the ASP web server is the backend web interface for user requests— e.g., get 
billing information, subscribe to a new app, or request a list of all possible apps a user 
can subscribe to. In the current model, the web server doesn't actually handle these 
requests, but instead passes them on to the appropriate eStream-centric server. 

NOTE: The following interfaces are not updated from the previous version! They 
were written with the assumption that the web server actually manages all the data 
described above. We need the server team to suggest the changes that should take 
place here! 

Functionality 

Interfaces 

AddADRMServerO 

The ASP Web Server is informed of the availability of a new ADRM server. The ASP 
Web Server adds this new ADRM Server to its Ust of online ADRM Servers. 
Periodically, the ASP Web Server can query the list of ADRM Servers for its load 
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information. When a new client connects to the ASP Web Server, the client can be 
informed of the subset of ADRM Servers with the least load. 

Callers: ASP Web Server 

Input: 

• ADRM Server IP 

• list of Account Ids that the ADRM Server supports. 
Output: 

• success or failure 
RemoveADRMServerQ 

The ASP Web Server is told of the removal of an ADRM Server. It must remove the 
ADRM Server from it's locally cached list of ADRM Servers to prevent any future 
clients from using that particular ADRM Server. 

Caller(s): ASP Web Server 

Input: 

• ADRM Server 
Output: 

• success or failure 
ValidateSubscribedUserQ 
Inputs: 

• SubscriptionToken 
Outputs: 

• success/failure 

Validate subscribed user is called by the ADRM server to check out a license. 

• Find account(accountNumber) 

• Find user(usemame) 

• Find license(SubscriptionlD) 
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If a license is found to be available, check it out and return OK else return 
NO_LICENSE_AVAILABLE. 

Add/RemoveAccountQ 

Input: 

• ownerUsemame 

• ownerPasswd 

• billinglnfo 

Output: 

• accountNumber 

Creator must supply info for the first user, who is also the account owner. 

Add/RemoveSubscribableAppQ 

Input: 

• application 

• AppServer locations 

• name description 

AddAccountUserQ 
Input: 

• account number 

• user name 

• initial password 

Add/Remove/lncrement/DecrementFloatingLicenseQ 
Input: 

• account number 

• list of users 

• subscription 

• number available 

Add/RemovePerUserLicense() 
Input: 
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• account number 

• user 

• subscription 
CompileAccountUsageO 
Input: 

• account number 

Add up and report all the usage by members of the account. 

ClearAccountUsageQ 

FreeLicenseQ 

Input: 

• user 

• subscription 

Frees a license that had been previously checked-out by a user. 

GetUserPrivelegesQ 

Input: 

• account number 

• usemame 

Check privileges of a logged in user for the purposes of allowing user/subscription 
management and other account changes 

ShowCheckedoutLicensesQ 

GetAccountlnfoQ 

Input: 

• account number 

• usemame 

ListPossibleSubscriptionsQ 
Input: none 
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ListCurrentSubscriptionsQ 
Input: 

• accountNvimber 
CreateSubscriptionQ 
Input: 

• account number 

• license data (depends on license type) 

• ApplicationPackage 

Output: 

• subscriptionID 

• ADRM server names. 

Builder 

Does not talk to any other module. Probably an associated set of tools managed via a 
script plus manual procedures that create intermediate data files, and finally produce the 
eStream set. 

Farm Manager 

Simply takes user commands and input, activating the following fimctions on the actual 
Application Server process: 

StopServer(boolean graceful) 

ConfigureServer(config_parameters) 

EnableEstreamSet(applD) - informs the app server that the set is ready to be served 
DisableEstreamSet(applD) - opposite of above 

Functionality required for upgraded Estream sets may extend this, management of farm- 
level stuff would be an extension of the above. Synchronizing the availability of apps 
between the Application Server and ADRM/Account DB must also be handled; at least 
initially, a human administrator should be able to flips the final switch. 
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Monitor 



The monitor utihty is responsible for monitoring the overall health of the system. It is 
responsible to report server status, server traffic, illegal access etc. It will pmg the 
Application Server and the ADRM server to gather the statistics and display them. 



Server data objects 

This section needs work! What should be here? 



eStream Set 

What is an eStream set? It consists of: 

• Page prediction map, indicates likelihood a page will be referenced successively 
after another page. Used to enable accurate prefetching by the client. 

• Spoofer info, stuff to initialize register & file spoofer to enable application to run 
on the client. (ITARD & "Spoofed file mapping list") 

. Application content. This includes all files of the application in possibly multiple 
subtrees (stuff from C:, Z:, Common Files, etc.). These are all then placed under 
a single application root directory which is "mounted" under the eStream file 
system under the application's directory ("Microsoft Office" or whatever). This 
directory structure is then processed to create a special eStream metadata file to 
represent it, and map file names to FilelDs, one assigned to each file for the app, 
used by the client. All of the application files can then be placed into a single 
large file with a FilelD index in front to allow the Application Server to map 
requested FilelDs to offsets in the large application content image file. 

• The above takes care of everything in the AppInstallBlock except for possible 
COM dlls that might be necessary (TBD what mechanism generates these). 



Account/Subscription Database: 



Description: 

The Account Subscription database manages all the data required to manage accounts, 
applications and subscriptions for an applications service provider. It is used to venfy 
users and subscription rights, to log usage, and compile billing for application rental. 1 
document describes the data model for the db, the list of accessor APIs, and finally, a 
of scenarios that exercise these APIs. 



Data Model 



The following is a description of the data model for the Account/Subscription Database. 
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There are two types of records described. 

Static Objects are objects that are stored persistently in the Account/Subscription 
database. The attributes of each record are divided into two categories: 

• Owns: specifies data that is actually contained in and managed by the object. 
These are the attributes that make the object what it is. For example: an account is 
not an account unless it has billing information. Each item (except the ASP) must 
have only one owner, but may have many references. 

• References: Attributes that create associations to other first-class objects, making 
navigation from one item to another simpler. 

Transient Objects represent data structures that are created in order to pass information 
from place to another. For the most part, they provide shorthand identifiers for static 
objects. 

Each attribute listed below has the format: 

Type: name - (optional) description 
For attributes that don't yet have a defined type, the type is undefined 

Static Data (Database records) 

ASP - The top-level container. 

Owns: 

• List<Account>: accounts - all of the accounts 

• List<Subscription>: subscriptions - all of the available subscriptions 

• List<Applications>: applications - all of the applications currentiy served by 
Application Servers 

References: 

• None 

Account - Collection of attributes that make up a single account: 

Owns: 

• Number: accountNumber - number that identifies this account 

• List<User>: users - all of the account users 

• Undefined: billinglnfo - Currentiy undefined type 
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• List<License>: licenses - all of the licenses (per-user and floating) that are 
managed by the account 

• List<UsageRecord> usage - list of records that describe usage of all account users 

References: 

• User: owner - a user who created the account - must be one of the users 
User - a single user of an Account 

Owns: 

• String: usemame 

• String: password 

• Undefined: Role - specifies permissions on the account, i.e. owner, administrator 
vs. regular user 

• Undefined: Userlnfo - Real name, contact info etc 

References: 

• List<License>: licenses currently held by user 

• List<License>: licenses - licenses to which the user has access. These might be 
either per-user or floating licenses, or a combination of the two. This list might 
also incorporate a means of specifying a preference - for example if a floating 
license and a per-user license are available, use the per-user 

Application - single application that has been made available on one or more application 
servers. 

Owns 

• String: name 

• String: description 

• Undefined: AppServer location(s) - A location may be a host name that must be 
resolved by the appserver farm, or may be an IP address. 

• Number: AppID 

• A list of FilelDs for this app. 

References: 

• None 

Subscription - An application or group of applications that have been made available for 
rental by a user. 
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Owns: 

• Number: SubscriptionID 

• String: name 

• String: description 

References: 

• User: user 

• List<Application>: applications - application(s) associated with this subscription 
ApplicationPackage - application(s) that can be rented 

Owns: 

• Undefined pricing 
References: 

• List<Applicati6n> applications 

License - base for other licenses - All licenses support the same APIs - check out, check 
in 

Owns: 

• None 
References: 

• Subscription: subscription - subscription for which this license grants rights 

FloatingLicense - one of a fixed number of licenses distributed to a list of users on a 
first-come first-serve basis. Contains all attributes in License plus: 

Owns: 

• Number: numTotal 

• Number: numlnUse 

References: 

• List<User> holders - the current holders of this license (length = numlnUse) 

• List<User> allowedUsers - list of users allowed to check out this license 
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PerUserLicense - license tied to a particular user -one desktop (machine) at a 
time. Contains attributes in license plus: 

Owns: 

• Boolean: isInUse 

• Undefined - desktopID 

References: 

• User: allowedUser - the (only) user allowed to pull this license. 
UsageRecord - Describes a billable use of the application 

Owns: 

• Undefined: Start time 

• Undefined: End time 

References: 

• Subscription: subscription 

• User: user 

Transient Data 

AccountNumber - integer that uniquely identifies a single account with an application 
service provider. 

UserName - shing that uniquely identifies a single user within an account. To uniquely 
specify a user to an ASP, it is necessary to qualify the UserName with the 
AccountNumber of which he is a member. All users have 

UserVerifier - combination of the AccountNumber, UserName, and UserPassword. 
Uniquely identifies a user within an ASP 

• AccountNumber 

• UserName 

• UserPassword 

SubscriptionID - Integer that uniquely identifies a subscribable application or collection 

of applications within an ASP. 

SubscriptionToken - describes a user-subscription to the ADRM server. Identifies the 
subscription as well as the user trying to access it. 
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• UserVerifier 

• SubscriptionID 

AppID - A unique numeric representation of each eStreamed application. For example, 
word := 1000, excel := 1001, office := 1002 etc. We should be able to represent software 
packages using AppIDs. 

FilelD - Within an eStreamed app, each app-file gets a unique numeric ID. 
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eStream Web Server Load Monitoring Applet Low Level 
Design 

Jae Jung 
10/27/00 
Modified: 10/30/00 



Functionality 

One of the requirements for the eStream web server is a facihty for monitoring 
server load (eStream requirements 3.0, 3.2, 3.4, 3.5). Per this document, this 
facihty will be provided by a graphical load-monitoring applet that will be 
available for deployment at customer sites as part of the eStream web server 
installation. 

The load-monitoring applet will present information in the following formats 
through a graphical interface: 

• Real-time server load information plotted on strip chart 

• Historical server load information plotted on line chart 

• Multi-server real-time or historical load information on a line chart 

Requirements 

The following Ust details the provisional requirements for the load-monitoring 
applet. The remainder of this design document is based on these requirements. 

1 . The applet will be able to display server loads in "real-time" as load data is 
retrieved from the server. 

2. The applet will be able to display (in a separate mode from real-time 
monitoring) historical load information to the extent that this information is 
available in the database. 

3 . The applet will be configurable (via applet parameters) for the following 
settings: 

~ Data retrieval rate, i.e., the frequency with which the applet request new 
load data from the web application server 

~ Chart window size, i.e., the number of datapoints shown in the chart 
window at any one time. 

4. The applet will be capable of concurrently displaying the load of multiple 
servers in a clear and concise fashion. 

5. The applet will support the displa)dng of cumulative and average load 
statistics for multiple servers. 



6. In real-time mode, the applet will operate as a strip chart, with the fastest chart 
speed determined by a global configuration setting in database, typically 
corresponding to the fi:equency with which the eStream Monitor inserts load 
records into the database. 

7. The applet will retrieve load information from the database via an http 
connection to the eStream web app server. 

8. The applet will run in browsers with Java 1 .0.x and 1 . 1 .x support. 

9. Internationalization support. Both the Applet and the backend pieces should 
be intemationalizable. 



Description 

The load monitoring applet is comprised of two components. The first 
component will manage the retrieval of real-time or historical server load 
information firom the eStream database. The second will consume this data and 
present this information to the user in a clear and concise graphical format. In 
addition to the applet, server-side objects will need to be written or extended to 
service data requests from the applet. 

For the alpha-release of the eStream web server, the graphical presentation 
component will not be written internally; rather a commercially available applet 
or package will be used. Other aspects of the applet and the server-side 
components will be implemented to facilitate transitioning to an internally 
developed graphical component if such a decision is made for later releases. 

User Interface Design 

The following two screen shots are representative of the way that the load 
monitoring applet might be used in a particular monitoring/administration Brower 
interface. The first shot shows a general server administration and monitoring UI 
and the second shows a detailed load monitoring UI within which the load 
monitoring applet will be embedded. 

The UI options shown in the second shot illustrate some of the reporting options 
for which the load monitor will be initially configurable. The applet(s) will be 
readily extensible to generate additional reports and more complex data 
combinations if desirable. 



Interface Definitions 

The load monitoring applet has two interfaces with other webserver components: 
<APPLET> tag parameters for its configuration within the web browser page and 
the HTTP request/response format with the web application server to request and 
receive load data. 



1 . Applet <-> HTML Page Interface (<APPLET> Tag Parameters): 



Parameter 


Req'd? 


Significance 


Default 


hostName 


Yes 


Host name of webserver 


None 


hostPort 


No 


Host port of webserver 


"80" (int) 


chartSpeed 


No 


Time (in seconds) between chart scroll; also 
the resolution of the x-axis 


"5"(int) 


updatePeriod 


No 


hiterval (in seconds) between HTTP requests 
for server load data. Note if chartSpeed < 
updatePeriod, the applet buffers load data for 
presentation. Maximum information latency 
will at most be equal to this value plus round 
trip time for the data request. 


"10" (int) 


chartWidth 


No 


Width (in ticks) of the applet chart; in 
conjunction with chartSpeed, implicitly 
determines the amount of data displayed 


"50" (int) 


mode 


No 


"current" for real-time monitoring and 
"history" for historical 


"current" 


startDate 


No 


if mode="history", the starting date for 
which the historical load chart will be 
plotted. Note if mode- 'current", this 
parameter is ignored. 


none; value 
must be in Java 
Date format 


stopDate 


No 


if mode="history", the stop date for which 
the historical load chart will be plotted. Note 
if mode="current", this parameter is ignored. 
Note that stopDate override the chartSpeed 
setting insofar as x-axis resolution is 
concerned. 


none; value 
must be in Java 
Date format 


numServers 


Yes 


number of servers to be plotted 


1 (int, max 50) 


serverNamel 
serverName2 

serverNameSO 


Yes 


the id of the server(s) being plotted (up to 50 
servers can be plotted at once). This must 
correspond to a existing serverlD in the 
Server database table 


none 


serverDatal 
serverData2 

serverDataSO 


No 


an initial set of data with which to display 
the initial chart. Note that these parameter(s) 
are the default means by which historical 
load data is displayed. Each set should be a 
comma dehmited hst of numbers (float) 


none 



• Note that applet tag parameters are all strings; where the applet does an 
internal type conversion, the target type format of the particular parameter 

has been noted. 

• Note that the parameters determining the appearance of the chart (e.g., line 
colors, grid painting options, custom labels, etc.) are not included in this 
Ust. These parameters will either be determined by the parameters 
available for configuration in the commercial charting applet or TBD at 
some later date if the charting component is developed intemally. 

2. Applet <-> Web App Server Interface (HTTP) 

Input; 

The load monitoring applet will request load data from the webserver by 
executing the web server's MonitorServlet with the following parameters: 

href="/MonitorServlet?action=getLoadData 
&serverld=... 
&startDate=... 
&stopDate=... 
&numPoints=... 

where: 

• serverld is a comma-delimited hst of one or more server(s) for which load 
information is being requested. Note that this serverld corresponds to the 
serverlD attribute in the Server database table. 

• startDate is the (exclusive) starting date (in Java Date format) for which 
the load information is being requested 

• stopDate is the (inclusive) end date (in Java Date format) for which the 
load information is being requested 

• numPoints is the number of data points requested between the start and 
stop dates. 

The applet will process the retum data points as equally time-spaced points 
between the requested start and stop dates. 

In addition, if startDate=stopDate, only one load data point (i.e., the most recent) 
will be retumed by the servlet. 

Output; 

The load monitoring applet will expect load information from the web server via 
HTTP response in the following XML-like format: 



<loadData> 



<serverid=a > 



<LOADLIST> 



<LOAD 
<LOAD 
<LOAD 



value=xl/> 
value=x2/> 
value=x3/> 



<LOAD value=xN/> 
</LOADLIST> 

</server> 
<server id=b> 
<LOADLIST> 



<LOAD value=yN/> 
</LOADLIST> 

</server> 
</loadData> 

In the above example, xl to xN represents load values for the server a (by 
serverDD), and yl to yN represents load values for server b. 



Testing Design 

The load monitoring applet and related server-side Java code will need to be tested 
according to the plan outlined for other web server components in the Web 
Server/Database Low Level Design (WebServerDB-LLD.doc). Additionally, it should be 
noted that certain applet-related parameters (i.e., updatePeriod) that affect the frequency 
with which the applet requests load data from the web server are good candidates for 
tuning in order that a good balance between Ul/measurement response and web server 
response performance be struck. 

Upgrading/Supportability/Deployment Design 

Upgrading/Supportability/Deployment Design will follow the model described for the 
Web Server in the Web Server/Database Low Level; Design (WebServerDB-LLD.doc). 

Open Issues 



<LOAD 
<LOAD 
<LOAD 



value=yl/> 
value=y2/> 
value=y3/> 



How much will it cost to deploy the chosen commercial java applet/package(s) as an 
OEM installation? We need to find this out before we decide on a commercial 



package. Also, another criteria for the choice would be: will we get support. Do we 
get the source code too? 
2. Longer term, where' s the cost/benefit breakpoint for the above where it makes more 
sense to write our own charting applet/package? 



